1. Field of the Disclosure
The present disclosure relates to implementing learning in spiking neuron networks.
2. Description of Related Art
Spiking Neural Networks
Artificial spiking neural networks are frequently used to gain an understanding of biological neural networks, and for solving artificial intelligence problems. These networks typically employ a pulse-coded mechanism, which encodes information using timing of the pulses. Such pulses (also referred to as “spikes” or ‘impulses’) are short-lasting (typically on the order of 1-2 ms) discrete temporal events. Several exemplary embodiments of such encoding are described in a commonly owned and co-pending U.S. patent application Ser. No. 13/152,084 entitled APPARATUS AND METHODS FOR PULSE-CODE INVARIANT OBJECT RECOGNITION”, filed Jun. 2, 2011, [client reference BRAIN.001A] and U.S. patent application Ser. No. 13/152,119, Jun. 2, 2011, entitled “SENSORY INPUT PROCESSING APPARATUS AND METHODS”, [client reference BRAIN.003A] each incorporated herein by reference in its entirety.
A typical artificial spiking neural network, such as the network 100 shown for example in FIG. 1, may comprise a plurality of units (or nodes) 102, which correspond to neurons in a biological neural network. Any given unit 102 may receive input via connections 104, also referred to as communications channels, or synaptic connections. Any given unit 102 may be connected to other units via connections 112, also referred to as communications channels, or synaptic connections. The units (e.g., the units 106 in FIG. 1) providing inputs to any given unit via for example connections 104, are commonly referred to as the pre-synaptic units, while the unit receiving the inputs (e.g., the units 102 in FIG. 1) is referred to as the post-synaptic unit. Furthermore, the post-synaptic unit of one unit layer (e.g. the units 102 in FIG. 1) can act as the pre-synaptic unit for the subsequent upper layer of units (not shown).
Individual ones of the connections (104, 112 in FIG. 1) may be assigned, inter alia, a connection efficacy (which in general refers to a magnitude and/or probability of influence of pre-synaptic spike to firing of post-synaptic neuron, and may comprise, for example a parameter: synaptic weight, by which one or more state variables of post synaptic unit are changed). During operation of the pulse-code network (e.g., the network 100), synaptic weights may be typically adjusted using what is referred to as the spike-timing dependent plasticity (STDP) in order to implement, among other things, network learning.
One such adaptation mechanism is illustrated with respect to FIGS. 2-3. Traces 200, 210 in FIG. 2 depict pre-synaptic input spike train (delivered for example via connection 104_1 in FIG. 1) and post synaptic output spike train (generated, for example, by the neuron 102_1 in FIG. 1), respectively.
Properties of the connections 104 (such as weights w) may be adjusted based on relative timing between the pre-synaptic input (e.g., the pulses 202, 204, 206, 208 in FIG. 2) and post-synaptic output pulses (e.g., the pulses 214, 216, 218 in FIG. 2). One typical STDP weight adaptation rule is illustrated in FIG. 3, where rule 300 depicts synaptic weight change Δw as a function of time difference between the time of post-synaptic output generation and arrival of pre-synaptic input Δt=tpost−tpre. In some implementations, synaptic connections (e.g., the connections 104 in FIG. 1) delivering pre-synaptic input prior to the generation of post-synaptic response are potentiated (as indicated by Δw>0 associated with the curve 302), while synaptic connections (e.g., the connections 104 in FIG. 1) delivering pre-synaptic input subsequent to the generation of post-synaptic response are depressed (as indicated by Δw<0 associated with the curve 304 in FIG. 3). By way of illustration, when the post-synaptic pulse 208 in FIG. 2 is generated: (i) connection associated with the pre-synaptic input 214 may precede the output pulse (indicated by the line denoted 224) and it may be potentiated (Δw>0 in FIG. 3 and the weight is increased); and (ii) connections associated with the pre-synaptic input 216, 218 that follow may be depressed (Δw<0 in FIG. 3 and the weights are decreased).
Spiking Neuron Models
Generalized dynamics equations for spiking neurons models may be expressed as a superposition of input, interaction between the input current and the neuronal state variables, and neuron reset after the spike as follows:
                                          ⅆ                          q              →                                            ⅆ            t                          =                              V            ⁡                          (                              q                →                            )                                +                                    ∑                              t                out                                                                                  ⁢                                                  ⁢                                          R                ⁡                                  (                                      q                    →                                    )                                            ⁢                              δ                ⁡                                  (                                      t                    -                                          t                      out                                                        )                                                              +                                    G              ⁡                              (                                  q                  →                                )                                      ⁢                          I              ext                                                          (                  Eqn          .                                          ⁢          1                )            where:                {right arrow over (q)} is a vector of internal state variables (e.g., comprising membrane voltage);        Iest is external input into neuron;        V is the function that defines evolution of the state variables;        G describes the interaction between the input current and the state variables (for example, to model postsynaptic potentials); and        R describes resetting the state variables after the output spikes at tout.        
For example, for IF model the state vector and the state model may be expressed as:{right arrow over (q)}(t)≡u(t);V({right arrow over (q)})=−Cu;R({right arrow over (q)})=ures;G({right arrow over (q)})=1,  (Eqn. 2)where C is a membrane constant, ures is a value to which voltage is set after output spike (reset value). Accordingly, Eqn. 1 may become:
                                          ⅆ            u                                ⅆ            t                          =                              -            Cu                    +                                    ∑                              t                out                                                                                  ⁢                                                  ⁢                                          (                                                      u                    refr                                    -                  u                                )                            ⁢                              δ                ⁡                                  (                                      t                    -                                          t                      out                                                        )                                                              +                      I            ext                                              (                  Eqn          .                                          ⁢          3                )            
For a simple neuron model, Eqn. 1 may be expressed as:
                                                        ⅆ              v                                      ⅆ              t                                =                                    0.04              ⁢                                                          ⁢                              v                2                                      +                          5              ⁢              v                        +            140            -            u            +                                          ∑                                  t                  out                                                                                              ⁢                                                          ⁢                                                (                                      c                    -                    v                                    )                                ⁢                                  δ                  ⁡                                      (                                          t                      -                                              t                        out                                                              )                                                                        +                          I              ext                                      ⁢                                  ⁢                                  ⁢                                            ⅆ              u                                      ⅆ              t                                =                                    -                              a                ⁡                                  (                                      bv                    -                    u                                    )                                                      +                          d              ⁢                                                ∑                                      t                    out                                                                                                          ⁢                                                                  ⁢                                  δ                  ⁡                                      (                                          t                      -                                              t                        out                                                              )                                                                                      ⁢                                  ⁢                                  ⁢                  where          ⁢                      :                                              (                  Eqn          .                                          ⁢          4                )                                                          ⁢                                                                              q                  →                                ⁡                                  (                  t                  )                                            ≡                              (                                                                                                    v                        ⁡                                                  (                          t                          )                                                                                                                                                                        u                        ⁡                                                  (                          t                          )                                                                                                                    )                                      ;                    ⁢                                          ⁢                                          ⁢                                                    V                ⁡                                  (                                      q                    →                                    )                                            =                              (                                                                                                                              0.04                          ⁢                                                                                                          ⁢                                                                                    v                              2                                                        ⁡                                                          (                              t                              )                                                                                                      +                                                  5                          ⁢                                                                                                          ⁢                                                      v                            ⁡                                                          (                              t                              )                                                                                                      +                        140                        -                                                  u                          ⁡                                                      (                            t                            )                                                                                                                                                                                                  a                        ⁢                                                                                                  ⁢                                                  (                                                                                    bv                              ⁡                                                              (                                t                                )                                                                                      -                                                          u                              ⁡                                                              (                                t                                )                                                                                                              )                                                                                                                    )                                      ;                    ⁢                                          ⁢                                          ⁢                                                    R                ⁡                                  (                                      q                    →                                    )                                            =                              (                                                                                                    c                        -                                                  v                          ⁡                                                      (                            t                            )                                                                                                                                                                          d                                                                      )                                      ;                    ⁢                                          ⁢                                          ⁢                                    G              ⁡                              (                                  q                  →                                )                                      =                          (                                                                    1                                                                                        0                                                              )                                                          (                  Eqn          .                                          ⁢          5                )            and a,b,c,d are parameters of the model.
Some algorithms for spike-time learning (especially, reinforcement learning) in spiking neural networks may be represented using the following general equation described, for example, in co-pending and co-owned U.S. patent application Ser. No. 13/487,499 entitled “STOCHASTIC APPARATUS AND METHODS FOR IMPLEMENTING GENERALIZED LEARNING RULES”, incorporated supra:
                                          ⅆ                                          θ                i                            ⁡                              (                t                )                                                          ⅆ            t                          =                  η          ⁢                                          ⁢                      F            ⁡                          (              t              )                                ⁢                                    e              i                        ⁡                          (              t              )                                                          (                  Eqn          .                                          ⁢          6                )            where:                θi(t) is an adaptation (learning) parameter of a synaptic connection between the pre-synaptic neuron i and the post-synaptic neuron j;        η is a parameter referred to as the learning rate, η can be a constant parameter or it can be a function of some other system parameters;        F(t) is a performance function; and        ei(t) is eligibility trace, configured to characterize relations between pre-synaptic and post-synaptic activity.        
An exemplary eligibility trace may comprise a temporary record of the occurrence of an event, such as visiting of a state or the taking of an action, or a receipt of pre-synaptic input. The trace marks the parameters associated with the event (e.g., the synaptic connection, pre- and post-synaptic neuron IDs) as eligible for undergoing learning changes. In one approach, when a reward signal occurs, only eligible states or actions may be ‘assigned credit’ or ‘blamed’ for the error. Thus, the eligibility traces aid in bridging the gap between the events and the training information.
Various implementations of eligibility traces exist for different types of learning (e.g., supervised, unsupervised, reinforcement) for deterministic and stochastic neurons. However, existing implementations do not appear to provide a common framework thereby requiring reformulating, learning rules (e.g., for determining the eligibility traces) for each implementation.
Accordingly, there is a salient need for a more efficient method and apparatus for implementing state-dependent learning comprising in spiking neural network comprising a generalized (canonic) form of eligibility traces that may allow common implementation of the updates.